4 /PKT> 



10/523521 



WO 2004/019152 



PCT/SG2002/000179 



DT01 Rec'd PCT/PTC q 4 FEB 2005 



10 



15 



20 



25 



METHOD AND SYSTEM FOR DETERRENCE OF U NAUTHORISED— 
r ~~~~ REUSE OF DISPLAY CONTENT 

Field of the invention 

The invention relates generally to the field of content display. In particular, it relates 
to the deterrence of unauthorised reuse of display content for display on computer 
screens. 

Background 

With the rapid growth of the Internet, more and more images are accessible to Internet 
users. However, not all providers of images on the Internet are willing to permit users 
to keep the images provided and displayed on screens, much less allow the users to 
distribute the images at will. Present network protocols such as the HyperText 
Transfer Protocol (HTTP) do not address the issue of protecting images or other 
display content such as text downloaded from websites or other remote resources. As 
a result, any user who has access to a website is able to save and reuse display content 
downloaded from that website. This is because a number of World Wide Web 
(WWW) browsers provide a save function allowing users to save a web page and a 
composer feature which the users can use to edit the saved display content A user can 
also type a Print Screen key typically found on the keyboard of a computer to perform 
screen dumps or capture snapshots of a series of images stored in the video buffer of 
the computer by duplicating the video buffer. Using programs that enable the user to 
create, view, and edit images, the user can then easily obtain the images by retrieving 
these snapshots. 

It is hence difficult to prevent unauthorised reuse of display content downloaded from 
or provided via a remote resource. A diligent pirate can reuse both text and images by 
re-typing the text and performing screen dumps of the images shown on the screen. It 
is therefore desirable to deter unauthorised reuse of display content by either forcing a 
pirate to spend -significant amounts of money and time to obtain display content, or 
degrading the quality of reused screen content. To this end, a number of patents and 
conventional technologies have been proposed which are described hereinafter. 
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In US Patent No. 5,905,505, a method and a system for copy protection of on-screen 
display of text are disclosed, in which embodiments described therein prevent screen 
dumps of text displayed on computer screens by first having the text transformed into 
a textual image. In a first embodiment, two random background images are generated 

5 and added to the textual image to produce two composite images. The two composite 
images are then alternately displayed on a screen to simulate the on-screen rendering 
of the textual image. In a second embodiment, the textual image is modified to 
produce two random bitmapped images. The two random bitmapped images are then 
alternately displayed for simulating the on-screen rendering of the textual image. 

10 Although in both cases the textual image is irretrievable by performing screen dumps, 
because only one of the pair of composite or random bitmapped images can be 
captured as a snapshot each instant, a user acquires a visual perception resembling the 
textual image by averaging each pair of composite or random bitmapped images as 
these are alternately displayed on the screen. However, the embodiments work 

15 adequately only for high contrast images such as textual images but not for display 
content with low contrast such as pictures. 

In US Patent No. 6,209,103, a method and an apparatus for preventing reuse of text, 
images and software transmitted via networks are disclosed, which involve the 

20 transformation of textual information into images. In one embodiment, text is 
transformed into a textual image. In a further embodiment, an image is divided into 
multiple sub-images with each sub-image being a component of the image or an 
interference pattern is incorporated into different locations of the image to form 
multiple interference images with each interference image having a pattern at a unique 

25 location. An applet then displays these multiple images in succession over intervals. 
With these multiple images a user can only acquire a visual perception which 
resembles the original image but cannot obtain such an image from capturing 
snapshot of the video buffer once. Unfortunately, a pirate can still recover the image 
by capturing the video buffer several times to obtain all of the multiple images by 

30 trial. Generally, the number of attempts the pirate need to make to capture all of the 
multiple images is small because the number of sub-images is restricted by the 
screen's display refresh rate, which is usually less than 100Hz. 
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In US Patent No. 5,881,287, a method and an apparatus for copy protection of images 
in a computer system are disclosed which prevent unlicensed transfers of image data 
from a video adapter to other storage means. This is done by the interception of data 
transfer requests made to an operating system and identifying whether an image 

5 requested is one to be protected. If the image requested is one that is marked for 
protection, any data transfer incorporating the protected image is obscured, blacked 
out or replaced with another specified pattern or message. However, such a protection 
system can be easily bypassed. A pirate can load a hook program which intercepts the 
request before the protection system intercepts the request. For instance, when a user 

10 types the Print-Screen key on a computer keyboard, a hook program can intercept the 
Print-Screen request and copies the video buffer into another storage medium before 
the protection system can black out the snapshot. 

In US Patent No. 6,208,735, a secure spread spectrum watermarking method for 
15 multimedia data is disclosed, which uses digital watermark technology to embed a 
watermark representing copyright into protected content so as to indicate ownership 
of the protected content. To remove the watermark, a pirate has to degrade the quality 
of the protected content. However, this method cannot prevent unauthorised reuse of 
the protected content. 

20 

In Microsoft's windows operating systems, an application program interface (API) 
such as the sethookQ function is typically provided for intercepting a computer 
keyboard event. When a user types the Print-Screen key on a computer keyboard of a 
computer, the keyboard event can be intercepted by a hook application based on the 

25 sethookQ function, which would prevent the default process of copying the content of 
the computer's video buffer onto a clipboard thus preventing reuse of information 
stored in the video buffer. The disadvantage of doing this is that the API sethookQ 
function is a standalone application which has to be executed with a local user 
priority. This protection system is therefore not applicable to mobile codes such as 

30 Java applets. 

There is therefore a need for a display content reuse deterrence system for addressing 
the foregoing problems. 
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Summary 

An image is shown on a computer screen in the spatial domain. The human visual 
perception system, however, is sensitive to the frequency domain. To involve visual 
perception by the human visual perception system in the deterrence of reuse of 

5 display content, a system is disclosed in which an image is transformed into the 
frequency domain with corresponding frequency components. For every frequency 
component, a random number sequence is generated and a variant of the random 
number sequence is generated based on the value of the frequency component. The 
arithmetic mean value of any predefined length of segment of the random number 

10 sequence preferably tends to 0. A number is selected from every random number 
sequence and a noise image formed using the value of this number. The image in the 
frequency domain is combined with the noise image to generate a synthesised image 
known as a noise-deteriorated image, which is then transformed back to the spatial 
domain. From every random number sequence, multiple numbers are selected to - 

15 generate multiple synthesised images for forming an image sequence. The image 
sequence is then displayed on the computer screen. 

In such a system, an input such as an original image X, is processed into an output 
which consists of a. sequence of noise-deteriorated images X k for rendering on the 
20 computer screen. 



During preparation, the system transforms X into m components /i,^,- • fm in the 
frequency domain F. The system then generates a noise number sequence n={ r\u 
r,2,"-} for each component^. r=l,2, m before generating the noise-deteriorated 
25 images. Alternatively, each number in the noise number sequences can be generated 
while generating the corresponding noise-deteriorated image. 

To generate the noise-deteriorated images, the system performs the following 
operations: 
30 for fe=l,2, ■•' 

for i=l,2, m 
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end 

where the noise-deteriorated image F k in frequency domain 
has m components f ik . 1=1,2, • • m; 
inverse-transform F k into X k ; and 
5 render X k to the computer screen 

end. 

In accordance with a first aspect of the invention, a method for generating a plurality 
of noise-deteriorated images from an image displayable on a computer display for 
10 deterring unauthorised reuse of the image is disclosed, in which the method 
comprising the steps of: 

transforming an image displayable on a computer display into a plurality of 
frequency components; 

generating a noise sequence for each frequency component in the plurality of 
15 frequency components, each noise sequence having a plurality of noise numbers; 

selecting, in relation to a first noise-deteriorated image, a first noise number 
from each noise sequence; 

deteriorating each frequency component in the plurality of frequency 
components using the first noise number from each noise sequence corresponding to 
20 each frequency component in the plurality of frequency components to form a noise- 
deteriorated frequency component in a first plurality of noise-deteriorated frequency 
components; and 

transforming the first plurality of noise-deteriorated frequency components 
into the first noise deteriorated image for displaying on the computer display. 

25 

In accordance with a second aspect of the invention, a system for generating a 
plurality of noise-deteriorated images from an image displayable on a computer 
display for deterring unauthorised reuse of the image, in which the system comprises: 
means for transforming an image displayable on a computer display into a 
30 plurality of frequency components; 



WO 2004/019152 PCT/SG2002/000179 . 

6 

means for generating a noise sequence for each frequency component in the 
plurality of frequency components, each noise sequence having a plurality of noise 
numbers; 

means for selecting, in relation to a first noise-deteriorated image, a first noise 
5 number from each noise sequence; 

means for deteriorating each frequency component in the plurality of 
frequency components using the first noise number from each noise sequence 
corresponding to each frequency component in the plurality of frequency components 
to form a noise-deteriorated frequency component in a first plurality of noise- 
10 deteriorated frequency components; and 

means for transforming the first plurality of noise-deteriorated frequency 
components into the first noise deteriorated image for displaying on the computer 
display. 

15 Brief Description of Drawings 

Embodiments of the invention are described hereinafter in detail with reference to the 
drawings, in which: 

Fig. 1 is block diagram of a computer system for displaying an image according to an 
20 embodiment of the invention; 

Fig. 2 is a flowchart of an image transformation process according to the embodiment 
of Fig. 1; 

25 Fig. 3 is a flowchart of a process for the generation of interference images according 
to the embodiment of Fig. 1; and 

Fig. 4 is a block diagram of the image transformation and interference image 
generation processes of Figs. 2 and 3 for processing a two-by-two (2x2) pixels image 
30 for display. 
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Detailed Description 

Embodiments of the invention are disclosed hereinafter for addressing the need for a 
display content reuse deterrence system. 

5 In the field of vision sciences, the Talbot-Plateau Law states that the brightness of 
intermittent light is equal to the brightness of steady light with the same time- 
averaged luminance. For the Talbot-Plateau Law to apply, the intermittent light must 
illuminate at a rate beyond the Critical Flicker Frequency (CFF), which is the 
frequency at which flicker induced by intermittent illumination disappears. The 

10 Talbot-Plateau Law holds because when the human eye is presented with images 
changing at a rate above 24 frames per second, the human eye averages the images to 
visually perceive an averaged representation of the images rather than perceive the 
images separately. 

15 The embodiments described hereinafter aim to prevent reuse of display content while 
providing good visual perception quality. Measurements of vision perception quality 
vary because of different criteria for vision perceptual. The embodiments primarily 
focus on the criterion of screen flicker. 

20 Assume X is the original image, and Xk is an image at a k th frame shown on the screen 
in a sequence of T images which can be copied from a video buffer of a computer on 
which X is displayed by performing a screen dump. Generally, any display content 
reuse deterrence system should satisfy the following requirements: 



(1) Reduce to a minimum the arithmetic mean value of the difference Xk -X, Le. 



(2) At any frame k, the difference Xk-X should be at a maximum 

(3) If the screen display refresh rate is fast enough, the difference Xk - X should 
be global, i.e., distributed throughout the images. 



25 




30 



The first requirement aims to reduce screen flicker to a minimum so as to display an 
image of high quality to authorised users. The other requirements aim to introduce 
interference into the video buffer so that any screen dump cannot result in a high 
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quality image. The first requirement appears to contradict the others and a trade-off is 
therefore necessary for a display content reuse deterrence system to perform 
adequately. 

5 To meet all three requirements by the embodiments, a series of interference or noise 
images is added to the original image to generate a sequence of degraded or noise- 
deteriorated images which are rendered in succession and repeatedly in place of the 
original image. Although the noise-deteriorated images are displayed, an authorised 
user can visually perceive the original image with high quality. 

10 

All the pixels in an image are affected when the image is modified in the frequency 
domain. Applying interference to an image in the frequency domain can therefore 
satisfy the third requirement. 

15 There are two types of frequency components in an image, namely high and low 
frequency components. The high frequency components represent sharp or abrupt 
changes in the image while the low frequency components represent smooth or slow 
changes such as those found in a uniform region in the image. To meet the second 
requirement, high amplitude noise can be added to the high frequency components of 

20 the image to increase interference or noise in the image. 

The embodiments are described in detail hereinafter with reference to Figs. 1 to 4. In 
Fig. 1, a computer system in which embodiments may be implemented is shown. The 
computer system comprises of a Central Process Unit (CPU) 100, a keyboard 110 
25 with a key for activating screen dump, a video card 130 having a video buffer and a 
corresponding computer screen monitor 120, a storage unit 140 to store the program 
and data. Optionally, a network interface 150 is used to access to a network 160 to 
which the computer system is connected. 

30 Fig. 2 is a flowchart of an image transformation process according to an embodiment 
of the invention, whereby in a step 200, an original image is introduced into a reuse 
deterrence system consisting of the computer system shown in Fig. 1 and an 
appropriate reuse deterrence software application. In this process, the original image 
is to be transformed into a series of synthesised or noise-deteriorated images which 
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when displayed in succession give an authorised viewer a visual perception of the 
original image. The original image is first transformed into a series of frequency 
components in the frequency domain by a transformation process, such as Fast 
Fourier Transformation (FFT) or Discrete Cosine Transformation (DCT), in a step 
5 210. All frequency components are denoted asfi. For purposes of illustration, it is 
assumed that there are m frequency components where i = 1, 2, • , m. 

Corresponding to each frequency component.//, a random number sequence r,= {r,y, 
where j = 1, 2, ■•• } is generated. In order to generate a fc th synthesised image as 

10 display content for the £ th display frame on the monitor 120 in a sequence of 
synthesised images, a corresponding A* interference or noise image is generated by 
using tjk (i - 1, 2, ■ • • , m) in a step 215. Alternatively, a random number resource may 
be used in this step 215 to generate the k ih interference image. Thus the k 
synthesised image can then be generated by fi + where i = 1,2, , w, in a step 

15 220. 

In a step 230, the k^ synthesised image is transformed back to the spatial domain by 
reverse transformation. In a step 240, the spatial domain image of the A^ 1 synthesised 
image is provided as output to the monitor 160 for display as the k^ display frame in 
20 the sequence of display frames. 

In order to obtain a stable and flicker free image and prevent unauthorised reuse of the 
original image, the foregoing steps are preferably iterated indefinitely. Additionally, 
the random number sequence for each frequency component is preferably generated 
25 with the arithmetic mean value of each segment of the random number sequence of 
predefined length T tending to zero. This is because the predefined length T of each 
segment of the random number sequence for each frequency component is related to 
the noise amplitude, the frequency and perception tolerance of flicker. However, the 
smaller T is, the better the quality of the visual perception of the original image. 

30 

There are a number of ways to generate the interference images. Fig. 3 is a flowchart 
of an exemplifying process for the generation of the interference images. After 
receiving the original image in a step 300, a hash value v representing the original 
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image is calculated in a step 310. The hash value is a pseudo-random resource and 
can be obtained using SHA or MD5 hashing. 

In a step 320, a uniform-distribution random number sequence within interval (-1.0, 
5 1 .0) is generated from the seed of the hash, value v by using a random function such as 
rand( ). In a step 330, for each I th frequency component a new hash value h(y\i) is 
calculated, this new hash value marking the start position of a new random number 
sequence for the I th frequency component, where h( ) is a hash function. 

10 In a step 340, based on the amplitude of the I th frequency component and properties of 
the human visual perception system, the amplitude of the new random number 
sequence for the i th frequency component is adjusted to a suitable level. In a step 350, 
all noise components for the frequency components relating to the display frame 
are combined, forming the k tb noise image in the frequency domain. 

15 

The foregoing processes are more clearly described using an example shown in Fig. 4, 
which is a block diagram of the foregoing processes for processing a two-by-two 
(2x2) pixels image 410 for display on the monitor 160. 

20 Given the 2 x 2 pixels original image 410, a noise number sequence is generated as 
follows: 

It+T 

{R}={R { ,R2.R3,R4,R5fl6, where^tf, =0. 

25 The noise number sequence is then truncated into sub-sequences 420 and adjusted to 
appropriate amplitudes using a parameter {a,}. Each of the sub-sequences 420 also 
tends to zero if it is of sufficient length. These sub-sequences 420 are denoted as 
in}, {n}> {r 3 } and {r 4 }, 



30 a.\{R\, Ri t R3 t R4 f Rs t ~'} = { nu r\ 2 , r u , n 4 , n 5i nv } 

{r 2 }=a2{R2,Rz,R4,R5£6, •••} = { raw ^2, m, n** r 2 5,r 2 6, -- } 
{n}=o.3{R 3t R4 t R 5 Jl 6f Ry, —} = {r 3 i, r 32 , r n , r 34 , r 35 ,r 36 ,--- } 
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For purposes of illustration, it is assumed that the 5 th (£.6., fc=5) frame is to be 
rendered. A noise image 430 is created with elements from the sub-sequences 420 
5 corresponding to the ft* h display frame. The noise image 430 is added to the original 
image in the frequency domain 440 to form the ft* synthesised 450 image which is 
then transformed into the spatial domain. The ft th synthesised image in the spatial 
domain 460 can then be rendered on the monitor 160 during the ft* display frame. 



10 In the foregoing manner, a display content reuse deterrence system is disclosed. 
Although only a number of embodiments are described, it will be apparent to one 
skilled in the art in view of this disclosure that numerous changes and/or 
modifications can be made to such a system without departing from the scope and 
spirit of the invention. 

15 

In this system, an input such as an original imaged, is processed into an output which 
consists of a sequence of noise-deteriorated images X k for rendering on the computer 
screen. 

20 During preparation, the system transforms X into m components/i , j% % • • • , f m in the 
frequency domain F. The system then generates a noise number sequence ri={ r\u 
m>- • } for each component^. /=1,2, • • m before generating the noise-deteriorated 
images. Alternatively, each number in the noise number sequences can be generated 
while generating the corresponding noise-deteriorated image. 

25 

To generate the noise-deteriorated images, the system then performs the following 
operations: 

for ft=l,2, 

for /=1,2, m 

30 7*^+ Ik 



WO 2004/019152 PCT/SG2002/000179 

12 

where the k^ 1 noise-deteriorated image F k in frequency domain 
has m components f ik . z=l,2, m\ 
inverse-transform F k into X k ; and 
render X k to the computer screen 

end. 



